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Introduction 

• Killzone Shadow Fall is a First Person Shooter 

• PlayStation 4 launch title 

• In SP up to 60 characters @ 30 FPS 

• In MP up to 24 players @ 60 FPS 

• Gameplay logic has lots of 

• Branches 

• Virtual functions 

• Cache misses 

• Not suitable for PS3 SPU's but PS4 has 6 x86 cores 
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What do we cover? 

• What is an Entity? 

• What we did on PS3 

• Multi threading on PS4 

• Balancing Entities across frames 

• Performance issues found 

• Performance results achieved 

• Debug tools 
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What is an Entity? 


GDCEUROPE.COM 
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What is an Entity? 

• Base class for most game objects 

• E.g. Player, Enemy, Weapon, Door 

• Not used for static world 

• Has Components 

• E.g. Model, Mover, Destructibility 

• Update at a fixed frequency 

• 15, 30, 60 Hz 

• Almost everything at 15 Hz 

• Player updated at higher frequency to avoid lag 
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What is a Representation? 

• Entities and Components have Representation 

• Controls rendering, audio and VFX 

• State is interpolated in frames where entity not updated 

• Cheaper to interpolate than to update 

• Introduces latency 

• Always blend towards last update 
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Multi Threading Approach 
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PS3: 1 Entity = 1 Fiber 


yield fiber 

t 


resume fiber 

t 


PPU 


Entity 1 Entity 2 Entity 3 Entity 1 Entity 2 



• Most time spent on PPU 

• No clear concurrency model 

• Read partial updated state 

• Entities deadlock waiting for each other 
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PS4: 1 Entity = 1 Job 


CPU 1 


job 


Entity 1 Animation Entity 1 


CPU 2 

Entity 2 

Ray Cast 

Entity 2 


CPU 3 

Entity 3 



Time 


• No fibers 

• Entity updates as a whole 

• How to solve race conditions? 




GAME DEVELOPERS CONFERENCE'” EUROPE 2014 AUGUST 11-13, 2014 


Strong Dependencies 

• Make update order explicit: 



Ml* 


T 


strong 

dependency 


Missile Launcher 
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Soldier 


Time 


Missile 
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Missile Launcher Missile 
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Non-dependent Entities can Execute Concurrently 


Soldierl 




Weaponl 


Soldier2 


T 



•It] 


CPU 1 

Soldierl 

Weaponl 

CPU 2 

Soldier2 

Weapon2 


Time 


• No (indirect) dependency = no access 

• Works two ways: Weapon can access Soldier too 

• Create dependency has 1 frame latency 

• Global systems need locks 
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What about this? 




Soldierl Pistoll Soldier2 Pistol2 Bullet System 


Time 


• A few entities cause huge bottleneck 
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Non-exclusive Dependencies 



CPU 1 


CPU 2 


Soldierl Pistoll 

Soldier2 Pistol2 


Time 


Bullet System 


Barrier 


• Access to 'Bullet System' must be lock protected 
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Weak Dependencies 

• 2 tanks fire at each other: 




| Tankl 

Missilel 

Tank2 

Missile2 

or 

* Tank2 

Missile2 

Tankl 

Missilel 


Time 


• Update order reversed when circular dependency occurs 

• Not used very often (< 10 per frame) 
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Non-updating Entities 

• Entity can skip updates (LOD) 

• Entity can update in other frame 


not updating 



Entityl Entity3 


Time 


• Do normal scheduling! 
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Summarizing Dependencies 


Weak Strong 
Exclusive Non-excl. 


Symbol ► ► — » ------ 

Two way access / / / / 

Order guaranteed / / 

Allow concurrency + + ++ + + 

Require lock / / 


Weak 

Non-excl. 


Strong 

Exclusive 
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Referencing Entities 

• Dev build: CheckedPtr<Entity> 

• Acts as normal pointer 

• Check dependency on dereference 

• Retail build: Entity * 

• No overhead! 

• Doesn't catch everything 

• Can use pointers to members 

• Bugs were easy to find 
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Working with Entities without dependency 

• ThreadSaf eEntitylnterf ace 

• Mostly read only 

• Often used state (name, type, position, ...) 

• Mutex per Entity 

• Send message (expensive) 

• Processed single threaded when no dependency 

• Schedule single threaded callback (expensive) 

• Everything can be accessed 
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Scheduling Algorithm 
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Scheduling Algorithm 


not updating 
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Entities with exclusive dependencies merged to 1 job 

• Dependencies determine sorting 


• Non-exclusive dependencies become job dependencies 

• Expensive jobs kicked first! 
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Scheduling Algorithm - Edge Case 




• Non cyclic dependency becomes cyclic job dependency 

• Jobl and Job2 need to be merged 
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Balancing Entities Across Frames 
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Balancing Entities Across Frames 

• Prevent all 15 Hz entities from updating in same frame 

• Entity can move to other frame 

• Smaller delta time for 1 update 

• Keep parent-child linked entities together 

• Weapon of soldier 

• Soldier on mounted gun 

• Locked close combat 
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Balancing Entities - In Action 


Time -m-even frame 
Time in odd frame 
(should be equal) 


' Civilian @ 15Hz 

— - \ — 1 + ^update even frame 

Civilian @ 15Hz 

update Flash %Ti switch odd / even 
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Performance Issues 
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Performance Issues 

• Memory allocation mutex 

• Eliminated many dynamic allocations 

• Use stack allocator 

• Locking physics world 

• R/W mutex for main simulation world 

• Second 'bullet collision' broadphase + lock 

• Large groups of dependent entities 

• Player update very expensive 
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Cut Scene 


l 


Civilian 1 


Problem 


Cut Scene 


i — — i 1 


Civilian 2 Player Camera 


• Cut scene entity requires dependencies 

• 10+ characters in cut scene creates huge job! 
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Cut Scene - Solution 



• Create sub cut scenes for non-interacting entities 

• Master cut scene determines time and flow 

• Scan 1 frame ahead in timeline to create dependency 
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Using an Object 

• Dependencies on useable objects not possible (too many) 

• Get list of usable objects 

• Global system protected by lock 

• 'Use' icon appears on screen 

• Player selects 

• Create dependency 

• Start 'use' animation 

• Start interaction 1 frame later (dependency valid) 

• Hides 1 frame delay! 
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Grenade 

• Explosion damages many entities 

• Creating dependencies not option (too many) 

• ThreadSaf eEntitylnterf ace not an option 

• Need knowledge of parts 

• Run line of sight checks inside update 

• Uses scheduled callback to apply damage 
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Performance Results 
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Performance Results - Synthetic 


Level 

Counts 

Dependencies 

Max 
Entities 
in Job 

Speedup 

Number 

Entities 

Updating 

Entities 

Number 

Flumans 

Strong 

Excl 

Strong 

Non-Excl 

5000 Crates (20 |js each) 

5019 

5008 

1 

12 

4 

13 

2.8X 

100 Soldiers (700 ps each) 

326 

214 

105 

212 

204 

19 

4.2X 

500 Flags (160 ps each) 

519 

508 

1 

12 

4 

13 

0^2X 


6 cores! 



5000 Crates 


100 Soldiers 


500 Flags 
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Performance Results - Levels 


Level 

Counts 

Dependencies 

Max 
Entities 
in Job 

Speedup 

Number 

Entities 

Updating 

Entities 

Number 

Humans 

Strong 

Excl 

Strong 

Non-Excl 

The Helghast (You Owe Me) 

1141 

206 

32 

71 

23 

20 

4. IX 

The Patriot (On Vectan Soil) 

435 

257 

44 

199 

107 

15 

4.3X 

The Remains (12p Botzone) 

450 

128 

14 

97 

44 

18 

3.7X 



The Helghast The Patriot The Remains 
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Game Frame - The Patriot 
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Game Frame - Global 






taw 
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Game Frame - 


Fix Weak’ Dependency Cycles 
Execute Jobs Zut Scene 


Entity U pdate 


_ Single Threaded Callbacks 

t "i 

S Cheap Entities (Destructibles) 


a 




•ntityllpdate- TA_CPT1 


| EntityUpdate - Sub_Sequef 


S. gEntityllpdate -||£ntityt)pdate - .||EntityUpdat||Ent...|E. 


Sold.erRep - VSA_Security_Trooper 


[Soldier Rep • VSASecurityTroo. 


I Soldier - VSA. 


SknnedMode Rep 


SkmedModelRep - MsgUp 


SkinnedM 


[virtual void AbilityPoseDe.. 

[void SkeletonAnimator:A 


[virtual void AbilityPoseD. 


| virtual void AbilityPoseOeforroerRep "BuildArurriationComm 


[virtual 


void SkeletonAmmator: 


void GraphProg’-amlnsla 


[i void GraphProgramlns. 


Executed const 


void Gr 


_S.,§ErUityUpdate - Su [EntityUpdate - . pnt>tyUpdate - S... || jEntit. 


| SoldierR. 


[virtual void 


IskmnedModel - MsgEntit JlskinnedModel - Mi 


[void Morph 
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MR::Netw... MR:: 


[static void PBDGraphlnt. J 


I it at r vt: d I’HnSmu at 
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Debug Tools 
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• Dependencies get complex, we use yEd to visualize! 
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Conclusions 

• Easy to implement in existing engine 

• Game programmers can program as if single threaded 

• Very few multithreading issues 
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Questions? 


jorrit@guerrilla-games.com 



